Enhanced staged event-driven architecture

ABSTRACT

The present invention is an enhanced staged event-driven architecture (SEDA) stage. The enhanced SEDA stage can include an event queue configured to enqueue a plurality of events, an event handler programmed to process events in the event queue, and a thread pool coupled to the event handler. A resource manager further can be coupled to the thread pool and the event queue. Moreover, the resource manager can be programmed to allocate additional threads to the thread pool where a number of events enqueued in the event queue exceeds a threshold value and where all threads in the thread pool are busy.

BACKGROUND OF THE INVENTION

[0001] 1. Statement of the Technical Field

[0002] The present invention relates to the field of scalabledistributed services, and more particularly to a staged event-drivenarchitecture (SEDA).

[0003] 2. Description of the Related Art

[0004] Scalable distributed services have become an important aspect ofsoftware design in the distributed software development field. Inparticular, the ability to scale a distributed application to process anenhanced load represents a significant challenge to the proliferationand popularization of distributed applications. In this regard, if thechallenge is left unmet, the problem of scalability certainly willretard the growth of distributed software applications, includingInternet based Web services. Accordingly, several performance optimizingtechnologies have been applied to the problem of distributed applicationscalability in order to address the foregoing challenge.

[0005] At the forefront of technological enhancements, the threadpooling concept has been applied to distributed application design so asto provide an enhanced level of concurrent processing of applicationtasks. Examples include the technologies disclosed in U.S. Pat. Nos.6,427,161 B1, 6,377,939 and 5,991,792, in addition to United StatesPatent Application Publication Nos. US 2001/0018701 and US 2001/0005853,and Control of Dynamic Threads Pool for Concurrent Remote ProcedureCalls, IBM Technical Disclosure Bulletin, Vol. 38, No. 05 (May 1995). Ineach of the foregoing publications, the number of threads available foruse in a thread pool can be increased or decreased depending upon theload experienced in the application. Yet, the use of dynamicallyconfigured thread pools alone cannot overcome performance deficienciesexperienced under particularly variable loads such as those experiencedin an application server.

[0006] To that end, the staged event-driven architecture (SEDA) had beendeveloped at the University of California, Berkeley to support massiveconcurrency demands while simplifying the construction ofwell-conditioned services. As documented in the seminal SEDA paper, MattWelsh, David Culler, and Eric Brewer, SEDA: An Architecture forWell-Conditioned Callable Internet Services, in the Eightieth Symposiumon Operating Systems Principles (SOSP-18), Chateau Lake Louise, Canada(Oct. 21-24, 2001), in SEDA, applications are configured with a seriesof interconnected, event-driven stages where each stage enjoys acoupling to a stage queue. Within each stage, a thread pool can beallocated to support processing within the stage. Through this uniquecombination of queued, event-driven stages and thread pools, servicescan be well-conditioned to load, preventing resources from beingover-committed when demand exceeds service capacity.

[0007] In this regard, the SEDA design makes use of a set of dynamicresource controllers to keep stages within their specified operatingrange despite large fluctuations in load. More specifically, eachresource controller can monitor the load experienced by an associatedstage. When it is determined that the load has increased beyond athreshold level, the thread pool can be augmented. Conversely, when itis determined that the load has decreased, the number of threads in thethread pool can be reduced. Similarly, in respect to the batching ofevents in the event handler of the stage, the number of events processedby each iteration of the event handler can be adjusted based uponmeasured processing throughput.

[0008] As a result of the SEDA design, several prototypes have beenassembled, including the Sandstorm™ application server which, thoughcoded in the Java programming language, demonstrates performance whichrivals that of C and C++ based application servers. Other prototypeapplications include the SSL and TLS protocol library, known in the artas “aTLS”, a Gnutella™ packet router, and Arashi™, a Web-based e-mailservice similar to Yahoo!™ Mail. Finally, the Haboob™ high-performanceWeb server also incorporates a SEDA design which, in consequence of suchdesign, outperforms both the Apache™ and Flash™ Web servers.

SUMMARY OF THE INVENTION

[0009] The present invention is an enhanced SEDA stage. The enhancedSEDA stage can include an event queue configured to enqueue a pluralityof events, an event handler programmed to process events in the eventqueue, and a thread pool coupled to the event handler. A resourcemanager further can be coupled to the thread pool and the event queue.Moreover, the resource manager can be programmed to allocate additionalthreads to the thread pool where a number of events enqueued in theevent queue exceeds a threshold value and where all threads in thethread pool are busy.

[0010] In one preferred aspect of the present invention, the thread poolcan be a logical thread pool coupled to a global thread pool. In anotherpreferred aspect of the invention, the SEDA stage can be configured tohave no threads in the thread pool-particularly where the underlyingfunction of the SEDA stage is considered trivial. To that end, the SEDAstage can include a callable event handler in which a maximum andminimum number of threads in the thread pool is set to zero. Finally, itis to be noted that the event handler of the SEDA stage can beincorporated as part of an application server or a Web server.

[0011] A method for managing a thread pool in a SEDA stage can includemonitoring both a number of events in a coupled event queue, and a levelof busyness of threads in a coupled thread pool. At least one additionalthread can be added to the coupled thread pool only when the number ofevents in the coupled event queue exceeds a threshold value, and whenthe busyness of the threads in the thread pool exceeds a thresholdvalue. In particular, in a preferred embodiment, at least one additionalthread can be added to the coupled thread pool only when the number ofevents in the coupled event queue exceeds a threshold value, and whenthe all threads in the thread pool are busy.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012] There are shown in the drawings embodiments which are presentlypreferred, it being understood, however, that the invention is notlimited to the precise arrangements and instrumentalities shown,wherein:

[0013]FIG. 1 is a schematic illustration of a SEDA stage which has beenconfigured in accordance with several aspects of the present invention;and,

[0014]FIG. 2 is a flow chart illustrating a process for managing alogical thread pool in the SEDA stage of FIG. 1.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] The present invention is an enhanced SEDA stage. The enhancedSEDA stage can be configured to manage a logical thread pool in whichnew threads are added to the pool only when a number of queue events inan associated stage queue exceeds a threshold value, and when alreadyallocated threads in the logical thread pool are busy. Notably, unlike aconventional SEDA implementation, the thread pool managed in the SEDAstage need not include threads runnables which are exclusive to the SEDAstage. Rather, the thread runnables can be exclusive to a global threadpool which can be shared among multiple SEDA stages. Finally, the SEDAstage need not include a thread pool at all. Rather, the SEDA stage canbe configured as a directly callable stage, particularly to accommodatemore trivial application functions.

[0016]FIG. 1 is a schematic illustration of a SEDA stage which has beenconfigured in accordance with several aspects of the present invention.The SEDA stage 100 can include an event queue 110 in which incomingevents can be enqueued pending processing in the SEDA stage 100. TheSEDA stage 100 further can include an event handler 120 configured toprocess one or more computing events. Notably, the event handler 120 canbe provided by the application which conforms to the SEDA design. TheSEDA stage 110 yet further can include a thread pool 130 of runnablethreads. In this regard, the runnable threads can be allocated to handlethe processing of events in the event handler 120.

[0017] Importantly, whereas in a conventional SEDA implementation eachevent-driven stage can be coupled to a unique thread pool, in accordancewith one aspect of the inventive arrangements, the SEDA stage 100 can becoupled to a global thread pool 150 which can provide the underlyingrunnable threads to one or more other SEDA stages, e.g. the SEDA stage100A illustrated in FIG. 1. In this way, each SEDA stage 100, 100Acoupled to the global thread pool 150 can maintain a logical thread poolwhich appears as an exclusive pool of runnable threads, albeit eachrunnable thread in the logical pool actually can be a thread in theglobal thread pool 150. Consequently, enormous overhead reductions canbe experienced in the SEDA conforming application.

[0018] Like a conventional SEDA implementation, the SEDA stage 100 canbe coupled to a resource manager 140. As in a conventional SEDAimplementation, the resource manager 140 can monitor the number ofevents enqueued in an associated event queue. When the number ofenqueued events in the event queue 110 exceeds a threshold value, anadditional thread can be added to the thread pool 130. Additionally, theresource manager 140 can monitor the batching ratio of the event handler120 as events are processed and passed to other event queues 110A, 110B,110C or other SEDA stages 100A, 100B, 100C. As the batching ratioexceeds a threshold value, the number of events processed concurrentlyduring a single iteration of the event handler 120 can be modified.

[0019] Significantly, in accordance with the present invention, andunlike a conventional SEDA implementation, upon detecting an excessivenumber of events enqueued in the event queue 110, the resource manager140 of the SEDA stage 100 can add an additional thread to the threadpool 130 only if all threads in the thread pool 130 are considered“busy”. In this regard, FIG. 2 is a flow chart illustrating a processfor managing a logical thread pool in the SEDA stage of FIG. 1.Beginning in block 210, the number of enqueued events can be measured.If so, in decision block 220, if the number of enqueued events exceeds athreshold value, it further can be determined in decision block 230whether all threads are busy.

[0020] In block 240, only if all threads are busy can a new thread beallocated to the thread pool. Otherwise, in block 250 a delay can beincurred before repeating the process. In this way, although the numberof enqueued events can exceed a threshold, adding a new thread in theface of otherwise idle threads can result in performance degradation.Consequently, through the conditional addition of a thread as in thepresent invention, an order of magnitude performance advantage can berealized over the conventional SEDA design.

[0021] Importantly, in a conventional SEDA implementation, all SEDAstages include a thread pool having at least one thread available foruse by the event handler 120. Thus, in a conventional SEDAimplementation, all SEDA stages are “threaded”, regardless of thetriviality of the underlying function assigned to the SEDA stage. In thepresent invention, however, not all SEDA stages need be threaded.Rather, any SEDA stage, such as the SEDA stage 100B in FIG. 1 can be acallable event stage having no threads. In this regard, the maximum andminimum threads assigned to the SEDA stage 100B can be set to zero. Bymaking the number of threads assigned to the thread pool 150 of thecallable SEDA stage 100B configurable, the SEDA stage 100B can bedefined as callable either during development or at deployment.

[0022] The present invention can be realized in hardware, software, or acombination of hardware and software. An implementation of the methodand system of the present invention can be realized in a centralizedfashion in one computer system, or in a distributed fashion wheredifferent elements are spread across several interconnected computersystems. Any kind of computer system, or other apparatus adapted forcarrying out the methods described herein, is suited to perform thefunctions described herein.

[0023] A typical combination of hardware and software could be a generalpurpose computer system with a computer program that, when being loadedand executed, controls the computer system such that it carries out themethods described herein. The present invention can also be embedded ina computer program product, which comprises all the features enablingthe implementation of the methods described herein, and which, whenloaded in a computer system is able to carry out these methods.

[0024] Computer program or application in the present context means anyexpression, in any language, code or notation, of a set of instructionsintended to cause a system having an information processing capabilityto perform a particular function either directly or after either or bothof the following a) conversion to another language, code or notation; b)reproduction in a different material form. Significantly, this inventioncan be embodied in other specific forms without departing from thespirit or essential attributes thereof, and accordingly, referenceshould be had to the following claims, rather than to the foregoingspecification, as indicating the scope of the invention.

We claim:
 1. A staged event-driven architecture (SEDA) stage comprising:an event queue configured to enqueue a plurality of events; an eventhandler programmed to process events in said event queue; a thread poolcoupled to said event handler; and, a resource manager coupled to saidthread pool and said event queue and programmed to allocate additionalthreads to said thread pool where a number of events enqueued in saidevent queue exceeds a threshold value and where all threads in saidthread pool are busy.
 2. The SEDA stage of claim 1, wherein said threadpool is a logical thread pool coupled to a global thread pool.
 3. TheSEDA stage of claim 1, wherein the SEDA stage comprises a callable stagein which a maximum and minimum number of threads in said thread pool isset to zero.
 4. The SEDA stage of claim 1, wherein the SEDA stagecomprises a callable stage having no threads in said thread pool.
 5. TheSEDA stage of claim 1, wherein said event handler is incorporated aspart of an application server.
 6. The SEDA stage of claim 1, whereinsaid event handler is incorporated as part of a Web server.
 7. A methodfor managing a thread pool in a staged event-driven architecture (SEDA)stage, the method comprising the steps of: monitoring both a number ofevents in a coupled event queue, and a level of busyness of threads in acoupled thread pool; and, adding at least one additional thread to saidcoupled thread pool only when said number of events in said coupledevent queue exceeds a threshold value, and when said busyness of saidthreads in said thread pool exceeds a threshold value.
 8. The method ofclaim 7, wherein said adding step comprises the step of adding at leastone additional thread to said coupled thread pool only when said numberof events in said coupled event queue exceeds a threshold value, andwhen said all threads in said thread pool are busy.
 9. The method ofclaim 7, wherein coupled thread pool is a logical thread pool andwherein said adding step comprises the step of adding allocating anadditional thread from a global thread pool for use by said logicalthread pool only when said number of events in said coupled event queueexceeds a threshold value, and when said busyness of said threads insaid thread pool exceeds a threshold value.
 10. The method of claim 7,wherein coupled thread pool is a logical thread pool and wherein saidadding step comprises the step of adding allocating an additional threadfrom a global thread pool for use by said logical thread pool only whensaid number of events in said coupled event queue exceeds a thresholdvalue, and when said all threads in said thread pool are busy.
 11. Amachine readable storage having stored thereon a computer program formanaging a thread pool in a staged event-driven architecture (SEDA)stage, the computer program comprising a routine set of instructions forcausing the machine to perform the steps of: monitoring both a number ofevents in a coupled event queue, and a level of busyness of threads in acoupled thread pool; and, adding at least one additional thread to saidcoupled thread pool only when said number of events in said coupledevent queue exceeds a threshold value, and when said busyness of saidthreads in said thread pool exceeds a threshold value.
 12. The machinereadable storage of claim 11, wherein said adding step comprises thestep of adding at least one additional thread to said coupled threadpool only when said number of events in said coupled event queue exceedsa threshold value, and when said all threads in said thread pool arebusy.
 13. The machine readable storage of claim 11, wherein coupledthread pool is a logical thread pool and wherein said adding stepcomprises the step of adding allocating an additional thread from aglobal thread pool for use by said logical thread pool only when saidnumber of events in said coupled event queue exceeds a threshold value,and when said busyness of said threads in said thread pool exceeds athreshold value.
 14. The machine readable storage of claim 11, whereincoupled thread pool is a logical thread pool and wherein said addingstep comprises the step of adding allocating an additional thread from aglobal thread pool for use by said logical thread pool only when saidnumber of events in said coupled event queue exceeds a threshold value,and when said all threads in said thread pool are busy.